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Overview 



Although a growing number of individuals arc enrolling in college in response to the increasing 
payoff to higher education, more than a third of them never finish. College completion rates are 
especially disappointing for low-income students, in many cases because they tend to enter college 
underprepared academically but also because they have more difficulty covering the costs of 
attendance. This report presents early results from a program at the University of New Mexico 
(UNM) that increases the financial support available to low-income entering students who enroll for 
a minimum number of credits and maintain a minimum grade point average. The program, called 
VISTA (Vision Inspired Scholarship Through Academic Achievement), is one of nine scholarship 
programs being tested across the country as paid of the national Performance-Based Scholarship 
Demonstration. The demonstration is testing several types of performance-based scholarships in 
order to identify promising strategics to increase college persistence and completion among low- 
income students. 

VISTA provides low-income entering freshmen with up to $1,000 in financial aid per semester for 
four semesters, in addition to any standard financial aid they receive. The funds are paid directly to 
the student in three installments each semester and are conditional on full-time enrollment and a “C” 
or better average. VISTA also provides students with enhanced academic advising, requiring them 
to meet at least twice during the semester with a designated VISTA advisor. 

The effects of VISTA are being assessed using a randomized control trial, in which over 1,000 low- 
income students who entered UNM in the fall of 2008 and the fall of 2009 were assigned at random 
to either the VISTA group, whose members are eligible for the program, or a control group, whose 
members arc eligible only for standard financial aid and counseling. The evaluation is tracking these 
students’ college performance for four years — that is, during the two years (four semesters) of the 
program and for two years after the scholarship ends. Early findings, through one year, indicate that 
although VISTA had no effects on credits or grades during the first semester, effects did emerge 
after that point: 

• VISTA encouraged students to attempt and earn more credits. Students in the VISTA 
group were substantially more likely than those in the control group to attempt 15 or more cred- 
its in the second semester, the minimum needed for VISTA. As a result, they were 8.8 percent- 
age points more likely to have earned 30 or more credits by the end of their first year, increasing 
the likelihood that they would be on track for an on-time graduation. 

• VISTA led to a net increase in financial aid dollars and allowed some students to reduce 
their reliance on loans. Students in the VISTA group received, on average, $900 more in aid 
than those in the control group, and were about 6 percentage points less likely to have loans. 

• Although VISTA did not affect overall enrollment rates for the third semester, it did result 
in students registering for more credits. About 78 percent of students in the study returned to 
UNM to register for classes in their third semester. Enrollment rates were similar for students in 
both the VISTA and control groups. However, VISTA students were much more likely to have 
enrolled for at least 15 credits. 

A final report on VISTA will be published in 2014. 
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Preface 



Although more and more high school graduates are enrolling in college these days, a large 
number of these students are either not finishing college or are taking a longer time to finish. In 
recent years, for example, less than two-thirds of 22-34-year-olds who had attended college had 
earned a degree. 

Completion rates are even lower for low-income students. While the funds that are 
available through the Pell Grant — the major source of financial aid for low-income students — 
have been expanded in recent years, today it covers only about a third of the costs of college. As 
a result, many low-income students resort to taking fewer classes and working while in school 
to supplement their aid, strategies that increase the risk of dropping out. 

Expanding aid to cover more of the costs of college has been suggested by some as a 
way to also increase college persistence — that is, reenrolling from semester to semester. In 
addition, findings from the Opening Doors demonstration — a multisite study of innovative 
interventions to help low-income community college students succeed academically — suggest 
that tying this aid to performance benchmarks might help students stay in school and accelerate 
their progress toward a degree. MDRC’s Performance-Based Scholarship Demonstration seeks 
to explore this possibility by testing variations of performance-based scholarships for different 
types of students in Arizona, California, Florida, New Mexico, New York, and Ohio. 

This report presents early findings from VISTA, a performance-based scholarship pro- 
gram at the University of New Mexico. VISTA, targeted to low-income entering freshmen, 
provides students with up to $1,000 per semester for four semesters, paid directly to the student 
in three installments, and enhanced academic advising. The additional aid, paid on top of any 
other aid for which the student is eligible, is conditioned upon the student maintaining full-time 
enrollment and a grade point average of at least 2.0. 

The early findings suggest that VISTA had no effects on credits or grades earned during 
the first semester, but encouraged more students to take and earn 15 or more credits during the 
second semester. Similarly, students who were eligible for VISTA were much more likely than 
control group students to enroll for 15 or more credits in their third semester. Finally, the extra 
money provided by VISTA enabled some students to reduce the amount they took out in loans. 

These first-year findings are an early look at VISTA’ s effects. As of the start of the sec- 
ond year, VISTA had encouraged more students to be on track for an on-time graduation. A 
future report will examine whether these early effects persist. 

Gordon F. Berlin 
President 
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Introduction 

In response to the rising payoff to higher education, more and more individuals now attend 
college. In 2009, for example, just over 70 percent of recent high school graduates were 
enrolled in college. 1 These high enrollment rates, however, have not translated into higher rates 
of college completion. 2 In 2000, less than 60 percent of 22- to 34-year-olds who had attended 
college had earned either an associate’s or a bachelor’s degree. 3 

Completion rates are even lower for low-income students, in many cases because they 
tend to enter college less prepared academically but also because they have more difficulty 
covering the costs of attendance than do students from families with higher incomes. Between 
two students entering college who are equally prepared academically, for example, the student 
from the lower-income family is significantly less likely to earn a degree. 4 The Pell Grant — the 
major source of aid for low-income students — covers only about one -third of the costs of 
college. 5 

Students from low-income families are thus left to supplement their aid by working 
while they attend college or taking out loans. A growing number of students are turning to loans 
to help finance college attendance, a trend that some view as troublesome, particularly for low- 
income borrowers who graduate with high amounts of debt. 6 Other students might attend a less 
expensive school, attend only part time, delay time to completion, or even drop out. The recent 
economic downturn has likely made it more challenging for these students to attend. Those with 
unemployed parents can count on less parental support, and those who typically work to support 
themselves may have more trouble finding a job. 7 Efforts have been made recently to increase 
aid for low-income students, and a large body of research finds that additional aid can increase 
college enrollment, although there is less evidence of its effects on persistence in college. 8 

This report presents early results from a program at the University of New Mexico 
(UNM) that provides financial support beyond standard financial aid packages for low-income 
entering students who enroll for a minimum number of credits and maintain a minimum grade 



'Snyder and Dillow (2010), Table 208. 

2 Tumer (2004). 

3 Dynarksi (2008). 

4 Bettinger (2004). 

5 Baum and Ma (2010). This figure refers to the cost of a four-year public institution. 

6 Baum and O'Malley (2003). Others would argue, however, that college remains a worthwhile investment 
for which to incur such debt and that the availability of student loans has helped many individuals attend college. 
7 Zemike (2010). 

8 See, for example. Long (2008); Dynarski and Scott-Clayton (2008); Richburg-Hayes et al. (2009a); 
Scott-Clayton (2009). 
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point average (GPA). The scholarship program, called VISTA, 9 provides students with up to 
$1,000 per semester for four semesters, paid directly to the student in three installments — the 
first payment at enrollment, the second payment at midterm, and the last payment at the end of 
the semester. VISTA also provides students with enhanced academic advising, requiring them 
to meet at least twice during the semester with a designated VISTA advisor, to verify, at a 
minimum, academic progress and eligibility for a payment. VISTA is one of several scholarship 
programs being tested across the country as part of the national Performance-Based Scholarship 
(PBS) Demonstration. The demonstration is being evaluated by MDRC, with UNM's primary 
funding coming from the Bill & Melinda Gates Foundation and the Open Society Foundations. 

The effects of VISTA are being assessed using a randomized control trial. Students who 
were found to be eligible for a Pell Grant who entered UNM as freshmen in the fall of 2008 and 
the fall of 2009 were assigned at random to a program group, whose members were eligible to 
receive the VISTA scholarship plus any other aid for which they were eligible, or to a control 
group, whose members were not eligible for VISTA but were still eligible for other financial aid. 
Because students were assigned at random to either research group, any statistically significant 
differences (differences that do not arise by chance) between them in college achievement and 
persistence can be attributed to VISTA. The evaluation is tracking these students’ college 
performance for four years — that is, for the two program years and two years after the scholar- 
ship program ends. This report is an early look at the program’s effects, covering students’ 
academic performance through the first year of college and their enrollment in the second year. 

• VISTA students earned about $1,500 in VISTA aid during Year 1 and 
about $900 more in total aid than control group students. 

The findings indicate that nearly all students who were eligible for VISTA — that is, 
those in the program group — succeeded in earning at least one payment during the first year, 
and more than 50 percent earned the full $1,000 in both semesters. VISTA transferred, on 
average, about $1,500 in aid to eligible students during Year 1. Also during Year 1, program 
group students as a whole received total financial aid that was about $900 more than students in 
the control group received, somewhat less than the $1,500 in VISTA aid, because the VISTA 
money enabled some students to reduce the amount they took out in loans. 

• The primary effect of VISTA through Year 1 is to encourage students to 
attempt and earn more credits. 

In terms of academic progress, VISTA had no effects on credits or grades earned during 
the first semester, but program group students were significantly more likely to attempt and earn 



9 Vision Inspired Scholarship Through Academic Achievement. 
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15 or more credits during the second semester (the minimum necessary to earn the scholarship). 
This effect occurred largely by encouraging students who would have taken and passed 12 
credits to take and earn 15 credits. Similarly, the latest data available indicate that program 
group students were more likely than control group students to enroll for 15 or more credits in 
their third semester. Students must earn an average of 16 credits per semester in order to 
graduate in four years. 

Thus, the primary effect of VISTA through Year 1 is to encourage students to attempt 
and earn more credits. The program did not have an effect on persistence from the first to the 
second year and has not had an effect on grades. It is encouraging that the program induced 
some students to take more courses without having a negative effect on grades. That VISTA 
allowed some students to reduce their reliance on loans might be viewed as a positive outcome 
as well, since studies indicate that debt accumulation can be a deterrent to finishing college, 
particularly for Hispanic students. 10 At the same time, program designers had expected that 
students would use the additional aid to cut back on their work hours and devote more time to 
school. If some students reduce their loan amounts instead, they may have difficulty meeting the 
academic performance requirements. 

These first-year findings reflect VISTA’s initial effects, since the scholarship is offered 
for four semesters. It is possible that these early effects on credit completion may grow over 
time, if staying on track in the early years contributes to performance in subsequent years. 



Overview of the Performance-Based Scholarship Demonstration 

The Performance-Based Scholarship Demonstration builds on promising results from Opening 
Doors, an earlier test of various strategies to increase college persistence and completion among 
low-income students. Opening Doors tested a range of such policies at community colleges 
across the country. The Louisiana site in the Opening Doors demonstration, which included two 
community colleges, provided students with a $1,000 scholarship for each of two semesters, or 
$2,000 total, paid in increments based on each student’s success in meeting key benchmarks. 11 
An evaluation of the program showed that the scholarship led to substantial improvements in 
credit accumulation and semester-to-semester persistence. 12 Moreover, some of these positive 
trends extended to the third and fourth semesters, when most students were no longer eligible to 
receive the scholarships. 



10 Burdman (2005). 

"The scholarship was given as follows: $250 was provided at enrollment, $250 upon passing midterms, 
and $500 upon passing courses. 

l: Richburg-Hayes et al. (2009a). 
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The PBS Demonstration, started in 2008, seeks to build on and replicate those encour- 
aging results. Currently, eight colleges and one intermediary (an existing scholarship provider 
or administrator that is not institutionally based) across six states are participating in the demon- 
stration. 13 Although the demonstration is being run in different types of institutions (for exam- 
ple, two-year colleges and four-year colleges) and targeting various types of students (for 
example, older students with children versus traditional entering freshmen), the common 
element across each institution is a scholarship paid in addition to other existing financial aid 
programs, in which students receive payments as they meet certain performance benchmarks, 
such as full-time enrollment, minimum midterm GPA, and minimum final-term GPA. 14 

VISTA stands out from the other PBS tests in several ways. First, it is one of two schol- 
arships targeting recent high school graduates at four-year colleges. Most of the other scholar- 
ships in the demonstration are targeted to older students attending community colleges, who 
tend to be older than 18 or 19. In addition, since the majority of low-income students attending 
UNM are Hispanic, VISTA also provides an important test of what might work to increase the 
relatively low completion rates among this group. And whereas many of the other PBS pro- 
grams offer a scholarship only, VISTA also includes an enhanced counseling component. 
Finally, VISTA is unique in that it is being offered on top of the New Mexico State Lottery 
Scholarship (discussed in greater detail below), which itself is a type of performance-based 
scholarship covering the cost of college tuition, although it is not paid directly to the student and 
does not come with enhanced advising. 



Students at the University of New Mexico and the 
Performance-Based VISTA Scholarship 

The University of New Mexico is the largest four-year college in New Mexico, with more than 
18,000 undergraduates and 6,000 graduate students at its Albuquerque Main Campus location. 
Reflecting the state’s population, the majority of UNM students belong to minority groups, and 
UNM is a U.S. Department of Education-designated Hispanic-Serving Institution. This desig- 
nation, for colleges where Hispanic students make up 25 percent or more of the undergraduate 
student body, means that UNM is eligible for federal grants that aim to expand educational 
opportunities for Hispanic students. 



13 The institutions include the Borough of Manhattan Community College and Eugenio Maria de Hostos 
Community College in New York; Lorain County, Owens, and Sinclair community colleges in Ohio; the 
University of New Mexico; Pima Community College in Arizona; Hillsborough Community College in 
Florida; and a state-level intermediary in California. 

14 See Richburg-Hayes et al. (2009b) for more information about the demonstration. 
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Table 1 gives a demographic and academic comparison of UNM students, Pell-eligible 
UNM students, and public college students nationally. UNM is clearly distinguished by its high 
minority enrollment. Hispanic students account for 38.4 percent of entering freshmen at UNM, 
compared with 9.4 percent, on average, at four-year public colleges nationally. Native Ameri- 
cans make up 4.6 percent of UNM entering freshmen, compared with 0.1 percent nationally. 

UNM students are typical among public college students nationally in terms of ACT 
scores and second-year retention, 15 but they graduate at slightly lower rates than students 
nationally; the six-year graduation rate at UNM is 42.5 percent, compared with 44.6 percent 
nationally. 

Pell-eligible students at UNM trail their more affluent peers on all academic measures 
except high school GPA. Most strikingly, 56 percent of Pell-eligible entering freshmen at UNM 
take at least one remedial class, compared with 43 percent of all UNM entering freshmen. At 
the same time, just over 70 percent of both all and Pell-eligible freshmen had a high school 
GPA of 3.0 or higher. This discrepancy suggests that high school GPA is at best a rough 
measure of academic preparedness among students. 

For students who remain enrolled, a smaller proportion of Pell-eligible students earn 
enough credits to make timely progress toward a degree; they trail all UNM students by 8 to 10 
percentage points in the first four semesters. Not surprisingly, the six-year graduation rate for 
Pell-eligible students is 8 percentage points lower than the graduation rate for all UNM students. 

In terms of affordability, UNM’s tuition and fees, at $5,150 for the 2010-2011 academ- 
ic year, are considerably lower than the $7,605 average for students who are state residents at 
four-year public colleges nationally. 16 In addition, a large majority of students at UNM benefit 
from a statewide scholarship funded by the New Mexico Lottery. The Lottery Scholarship pays 
tuition at any public college in the state beginning in the second semester of college enrollment 
and continuing for up to seven additional semesters. To qualify in a given semester, students 
must maintain a 2.5 cumulative GPA and earn at least 12 credit hours in all previous semesters. 
UNM offers incoming freshmen whose high school GPA is 2.5 or higher a Bridge to Success 
Scholarship (described in greater detail later in this report), which covers part of the first- 
semester tuition. About 75 percent of freshmen receive a Bridge to Success Scholarship. 
Nevertheless, when room and board, books, supplies, and personal expenses are added in, the 
cost of attendance exceeds $15,000, and the average unmet need (the cost of attendance minus 
financial aid and the expected family contribution) for Pell-eligible students exceeds $5,000. 



15 The ACT test assesses high school students’ general educational development and their ability to com- 
plete college-level work. 

16 Baum and Ma (2010). 
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The Performance-Based Scholarship Demonstration 
Table 1 

Characteristics of Incoming Freshmen at the University of New Mexico 
and Students in Four-Year Public Colleges 



University of New Mexico 



Characteristic 


Freshmen 
Entering UNM 
in 2006 and 2007 


Pell-eligible 
Freshmen 
Entering UNM 
in 2006 and 2007 


All Students 
in 4-year 
Public Colleges 
Nationally in 2004 


Family income $40,000 or less (Pell-eligible) (%) 


20.5 


100.0 


35.5 


Female (%) 


56.1 


59.5 


57.5 


Age (years) 


18.6 


18.5 


NA 


Race/ethnicity d (%) 


Hispanic 


38.4 


52.2 


9.4 


White 


45.8 


28.0 


66.8 


Black 


2.8 


3.6 


11.4 


Asian or Pacific Islander 


3.9 


5.0 


6.6 


Native American 


4.6 


7.2 


0.1 


ACT English' 5 


25th percentile score 


18 


16 


18 


75th percentile score 


25 


23 


24 


ACT Math 15 


25th percentile score 


18 


17 


18 


75th percentile score 


24 


23 


24 


High school cumulative GPA L 


3.3 


3.3 


NA 


High school cumulative GPA (%) 


3.5 to 4.4 


39.1 


38.5 


NA 


3 to less than 3.5 


33.2 


34.7 


NA 


2 to less than 3 


24.1 


24.8 


NA 


0 to less than 2 


0 


0 


NA 


No GPA available 


3.6 


1.8 


NA 


Placed in remedial English, reading, or math (%) 


43.1 


56.4 


NA 


Retention to fall semester, 2nd year (%) 


74.3 


70.2 


72.6 


Retention to fall semester, 3rd year 11 (%) 


58.3 


54.3 


NA 


Progress toward degree (for those still registered ) e (%) 


1st semester 


67.0 


58.2 


NA 


2nd semester 


51.5 


41.7 


NA 


3rd semester 


50.0 


41.0 


NA 


4th semester 


42.8 


35.0 


NA 


Six-year graduation rate f (%) 


42.5 


34.5 


44.6 



(continued) 
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Table 1 (continued) 



SOURCE: UNM Office of Institutional Research Freshman Cohort Tracking Database; NCES Integrated 
Postsecondary Education Data System (IPEDS) 2004 for national data. 

NOTES: Entering freshmen numbered 3,026 in 2006 and 2,910 in 2007. 

“Ethnic distribution is for 2005 entering freshmen. Distributions do not sum to 100 percent because some 
students declined to provide their ethnicity. 

b ACT scores range from 1 to 36. The median test taker who graduated from high school between 2008 
and 2010 earned a 20 in both the English and Math sections. The 25th percentile score was 15 for English and 
16 for Math and the 75th percentile score was 24 for both subjects. See 
http://www.actstudent.org/scores/normsl.html. 

“High school GPA ranges from 1.95 to 4.4. 
d This indicator is for 2006 entering freshmen. 

“Progress toward degree indicates that the student completed 12 or more credits in the first semester and 15 
or more credits every semester thereafter with a cumulative GPA of 2.0 or higher. 

Tor 2000-2002 entering freshmen. 



The VISTA Scholarship 

The VISTA Scholarship program aims to improve college outcomes and address sub- 
stantial unmet need for low-income students by providing up to $1,000 in financial aid per 
semester in addition to other aid, in increments tied to academic milestones. The scholarship is 
available for four consecutive semesters beginning with the student’s first semester at UNM. 
The milestone payments encourage students to make timely progress toward a degree. Students 
also receive enhanced academic advising. In particular, students are assigned to one advisor for 
the duration of the program and must receive early-semester and midsemester advising in order 
to earn interim payments. The milestones and payments are shown in Table 2. 

Students who meet the VISTA requirements will acquire at least 57 credits by the end 
of their fourth semester. Although registration for 12 credit hours is considered full time for 
financial aid purposes, students must average 16 hours per semester in order to graduate in four 
years. The 2.0 GPA required by VISTA is slightly higher than the cumulative 1.7 GPA over the 
first year required for freshmen to remain in good academic standing at UNM. UNM requires 
that students maintain a 2.0 cumulative GPA thereafter, the same as the VISTA requirement. 
Students who do not qualify for the midterm payment can recoup the full payment at the end of 
the semester if they improve their GPA or if they raise their earned credits or GPA for the 
semester by taking winter intersession or summer session courses. Students receive payments in 
any program semester during which they meet the milestones. 

The VISTA program should especially benefit students who fall below the 2.5 GPA re- 
quired by the Lottery Scholarship, but are still considered in good standing by the university 
with a GPA of 2.0. VISTA might also provide inducements for students who have a rough start 
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The Performance-Based Scholarship Demonstration 
Table 2 

Milestone and Payment Benchmarks 



University of New Mexico 



Semester 


Milestone 


Advising 

Required 


Payment 


1 


Registered for 12 or more credit hours at 21 -day census' 1 


Yes 


$250 




Enrolled in 12 or more credit hours, with a GPA of 2.0 or higher at midterm 


Yes 


$250 




Earned 12 or more credit hours with a GPA of 2.0 or higher for the semester 


No 


$500 


2 


Registered for 15 or more credit hours at 21 -day census 3 


Yes 


$250 




Enrolled in 15 or more credit hours, with a GPA of 2.0 or higher at midterm 


Yes 


$250 




Earned 15 or more credit hours with a GPA of 2.0 or higher for the semester 


No 


$500 


3 


Registered for 15 or more credit hours at 21 -day census 3 


Yes 


$250 




Enrolled in 15 or more credit hours, with a GPA of 2.0 or higher at midterm 


Yes 


$250 




Earned 15 or more credit hours with a GPA of 2.0 or higher for the semester 


No 


$500 


4 


Registered for 15 or more credit hours at 21 -day census 3 


Yes 


$250 




Enrolled in 15 or more credit hours, with a GPA of 2.0 or higher at midterm 


Yes 


$250 




Earned 15 or more credit hours with a GPA of 2.0 or higher for the semester 


No 


$500 



NOTE: a State funding for public universities is tied to enrollments at the 21-day census. 



at school to keep trying, since the program provides payments in any of the four semesters that a 
student meets the requirements. This feature contrasts with the less forgiving rules of the 
Lottery Scholarship, which once lost is difficult to regain. 

Tying aid to milestones that are associated with successful progress toward a degree 
provides incentives for students to take actions that make graduation more likely. Although it is 
also true that financial aid depends on enrollment, and the Lottery Scholarship provides incen- 
tives for students to make progress toward a degree, much financial aid — the Lottery Scholar- 
ship included — is opaque to students because it operates behind the scenes. Financial aid that 
covers tuition and fees involves a transfer from the financial aid office to the university rather 
than a direct payment to the student. While a student should, theoretically, be indifferent toward 
a reduction in amounts owed to the university versus a check in hand, economic experimental- 
ists and behavioralists have long appreciated the salience of actual, tangible cash in hand. The 
best explanation for this lack of equivalence between an accounting entry that reduces debt and 







a direct payment is “mental accounting.” 17 Mental accounting is the observed tendency of 
people to treat financial transactions and sources of income in a piecemeal way rather than in 
terms of their impacts on overall financial position. Thus, earned money carries a different 
propensity to consume than does “found” money; gamblers treat won “house money” different- 
ly from cash brought from home. 18 

In order to take advantage of the “cash in hand” incentive, VISTA payments are made 
directly to students. Students choose whether to receive their payment in the form of a check 
available for pick-up at the bursar’s office, or as a direct deposit into their bank account. The 
direct method of payment should make it clear to students that they, and not the university, are 
being rewarded for their performance. 

Enhanced Academic Advising 

The academic advising that VISTA students receive differs from standard student 
services in two ways. First, a VISTA student must see an advisor at least twice, and probably 
three times, during the semester: at or shortly after the 21 -day enrollment period, at midterm 
to verify the midsemester milestone, and later in the semester to register for the next semester. 
A non- VISTA student must see an advisor only once a semester, before registering for the 
next semester. 

Second, VISTA students always meet with the same academic advisor. Other students 
may request a particular advisor, but typically they see whoever is available at the time, much as 
a bank customer receives services from the next available teller. Although some VISTA 
advisors do not have a smaller case load per se (since they also see non- VISTA students), they 
are much more likely to get to know their VISTA advisees, since they see them consistently and 
more frequently. 

Of particular note is the midsemester milestone check. VISTA students should benefit 
from advising at midterm, when it is not too late to improve grades through tutoring, or to drop 
a class and register for a late-starting (second 8-week) class or a class during the winter interses- 
sion or summer session. 19 While any student may seek out an advisor at midterm, VISTA 
students have a financial incentive to do so. 



17 Thaler (1999). 
ls Thaler (1999). 

19 UNM offers some courses on an 8-week schedule. The courses carry the same academic credit as the 
standard 16-week courses, but are taught on a more condensed schedule. Students who are not passing a 
regular 16-week course can maintain minimum credit loads or minimum GPAs by registering for an 8-week 
course that meets during the second half of the regular semester. 
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In addition, because advisors get to know the VISTA students better, they are more like- 
ly to provide “holistic” counseling, which involves learning about — and potentially providing 
referrals for — nonacademic aspects of a student' s life, such as health, work, and family issues. 
Most students receive advising at the end of the semester, when they realize that they must see an 
advisor in order to register for the next semester. During these periods, advisors see literally 
hundreds of students, and are much less likely to employ holistic counseling. 

In short, VISTA is designed to guide students on a path toward completing the require- 
ments for their degrees. The 15-credit hour requirement should make graduation in four years 
more likely. Increased advising should help students take the courses they need to graduate. 
Increased monitoring should help students get the academic help they need before it is too late 
in the semester. 



The VISTA Evaluation 

The Random Assignment Process 

The random assignment of eligible students took place largely at the first-year student 
orientation sessions. These sessions, which span two days and occur once per week in July and 
August, are attended by all entering students. Students who are eligible for the PBS study were 
identified by the financial aid office as those who had completed a Free Application for Federal 
Student Aid (FAFSA) and were deemed eligible for a Pell Grant. 20 These students were sent 
letters prior to their scheduled orientation session and attended a separate VISTA scholarship 
session at some point during the orientation, in which they learned about the study, signed an 
informed consent form if they were willing to participate, and filled out the baseline question- 
naire. Completed questionnaires were then sent to MDRC, where a computer algorithm was 
used to complete random assignment. Students who were selected to be in the VISTA group 
were notified by regular mail and e-mail of their status, usually within a few weeks of their 
orientation session, and were told to come in during the first few weeks of class to meet with 
their VISTA advisor. Students who were selected for the control group were informed of their 
status by regular mail and e-mail, and were told that they were not eligible to receive the VISTA 
scholarship but would continue to receive all financial aid and student services that they would 
have received had they not participated in the study. In addition, these students were informed 
that their financial aid and access to counseling and other student services had not changed. 



20 An additional eligibility criterion was that the student had to have graduated from a New Mexico 
high school. 
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Data Sources 

This report uses data from several sources. First, two sources provide data for students 
prior to or at the point of study entry. Students completed a questionnaire called a Baseline 
Information Form (BIF) prior to entering the study, providing information about their age, 
race/ethnicity, employment status, and parents’ education levels. High school transcript data 
provided by the university contain information about students' performance in high school and 
test scores on college entrance exams. These baseline data are used to describe the study sample 
and, for future Performance-Based Scholarship reports, to define potential subgroups of interest, 
such as students with higher versus lower scores on college entrance exams. 

Second, the university provided MDRC with data on students' financial aid during the 
first two semesters of follow-up. These data include total aid awarded each semester and each of 
its components, including the New Mexico State Lottery Scholarship, the Pell Grant, other 
grants, subsidized and unsubsidized loans, and college work-study. Data were also obtained for 
students in the program group on the receipt of the VISTA scholarship at each disbursement 
period. These data are used to describe students’ success in earning the VISTA scholarship and 
how it contributes to and interacts with the type and amounts of students’ other financial aid. 

Finally, transcript data for each student were obtained from the university and contain 
detailed information for the first two semesters and the summer term on enrollment status, 
courses taken, credits earned, and grades. Data are also available on enrollment status at the start 
of the third semester. These data are used to assess the impact of the program on student 
progress and performance. 

Sample Characteristics 

The study sample for the VISTA evaluation includes about 1,100 Pell-eligible students 
who entered UNM as freshmen during fall 2008 and fall 2009. Table 3 presents selected 
characteristics of the full sample. 21 

Just over 60 percent of the sample is female, which reflects the Pell-eligible population 
in general at UNM. Since the program targeted first-time entering freshmen, nearly all the 
students are 17 to 18 years of age, and very few students are married or have children. A 
majority of the students are Hispanic (60.6 percent), and about 7 percent of students are Native 
American. 



“'Appendix Table A.l presents data for each research group, showing that the two groups look very simi- 
lar in terms of these characteristics measured at or before study entry. 
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The Performance-Based Scholarship Demonstration 
Table 3 

Selected Characteristics of Sample Members at Baseline 



University of New Mexico 



Characteristic 


Full 

Sample 


Female (%) 


60.8 


Age (%) 


17-18 years old 


93.7 


19-20 years old 


6.3 


Marital status (%) 


Married 


0.6 


Unmarried 


90.5 


Missing 


8.9 


Has one or more children (%) 


1.8 


Race/ethnicity a (%) 


Hispanic 


60.6 


White 


21.8 


Black 


2.7 


Asian or Pacific Islander 


3.5 


Native American 


6.9 


Other 3 


4.5 


Language other than English spoken regularly in home (%) 


22.0 


Diplomas/degrees earned 0 (%) 


High school diploma 


97.8 


GED certificate 


1.3 


Other 


1.2 


First person in family to attend college (%) 


32.8 


ACT English' 1 


25 th percentile score 


17 


75 th percentile score 


23 


ACT Math 01 


25 th percentile score 


16 


75 th percentile score 


23 


High school cumulative GPA 


3.3 


High school cumulative GPA (%) 


3.5 to 4.4 


38.2 


3 to less than 3.5 


33.9 


2 to less than 3 


24.6 


0 to less than 2 


0.0 


No GPA available 


3.3 



(continued) 
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Table 3 (continued) 



Characteristic 


Full 

Sample 


Highest degree/diploma earned by father (%) 


Not a high school graduate 


15.6 


High school diploma or GED certificate 


37.3 


Some college or associate's degree 


17.9 


Bachelor's degree or higher 


14.4 


Missing 


14.8 


Highest degree/diploma earned by mother (%) 


Not a high school graduate 


11.9 


High school diploma or GED certificate 


36.8 


Some college or associate's degree 


24.1 


Bachelor's degree or higher 


19.7 


Missing 


7.5 


Currently working (%) 


49.0 


Among those currently working, hours worked per week (%) 


1-10 hours 


5.9 


11-20 hours 


33.3 


21-30 hours 


29.9 


31-40 hours 


26.3 


More than 40 hours 


4.6 


Average hourly wage ($) 


8.2 


Plans to live on campus (%) 


42.9 


Sample size 


1,081 



SOURCE: MDRC calculations using Baseline Information Form (BIF) data and University of New Mexico 
placement test and high school data. 



NOTES: Italics indicate statistics calculated from a subset of the full sample. 

Missing values are not included in individual variable distributions. 

Distributions may not add to 100 percent because of rounding. 

“Respondents who said they are Hispanic and chose a race are included only in the Hispanic category. 
Respondents who said they are not Hispanic and chose more than one race are included in the Other category. 
b Other includes multiracial and other races/ethnicities. 

“Distributions may not add to 100 percent because categories are not mutually exclusive. 
d ACT scores range from 1 to 36. The median test taker who graduated from high school between 2008 and 
2010 earned a 20 in both the English and Math sections. The 25th percentile score was 15 for English and 16 
for Math, and the 75th percentile score was 24 for both subjects. See 
http://www.actstudent.org/scores/normsl.html. 
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In terms of academic performance, the students in the sample appear to be relatively 
prepared. Nearly 40 percent had a high school GPA of 3.5 or higher. In addition, the distribution 
of ACT scores is roughly similar to that of students nationwide (see Table 1). About a third of 
the students report that they are the first in their family to attend college, which is consistent 
with students' reports of their parents' education levels. For example, more than 50 percent of 
the students reported that their father had either not graduated from high school or had earned 
only a high school diploma or General Educational Development (GED) certificate. Finally, 
about half of the students were working at the time they entered the study, with about a third of 
these students working more than 30 hours per week. 



Implementing the VISTA Scholarship Program 

To implement the program, the VISTA site coordinator, within the University College Advise- 
ment Center, employed five VISTA advisors. Two of the advisors were linked directly to the 
scholarship. The remaining three were part of UNM's general advising program and were 
assigned to work with VISTA students along with their general advising load. For each entering 
cohort (students entering in fall 2008 and fall 2009), each of the two VISTA advisors was 
randomly assigned approximately 75 VISTA students, and the three other advisors were 
randomly assigned approximately 40 students in addition to their normal advising load. Thus, 
the student-advisor ratio for VISTA staff was 75 to 1 for the first year of the program and 150 to 
1 for the second and later years. This ratio is well below the ratio of 771 to 1 for UNM’s general 
counseling center (the University College Advisement Center, mentioned above), 22 and well 
below the ratio of 300 to 1 recommended by the National Academic Advising Association. 22 
These relatively low caseloads allowed the advisors to spend more time with their VISTA 
students to develop and foster a more holistic advising relationship. 

The First Milestone: Enrollment 

The first milestone was reached if the student was still enrolled at the 21 -day census, at 
which point students were required to come in for their first advising session. If they were 
enrolled for the necessary 12 or more credits (or 15 credit hours for the second through fourth 
semesters), students were either given a VISTA pass that allowed them to pick up their first 
check from the Cashier’s Office or had funds deposited directly into their accounts. The pass 
had to be signed by both the advisor and the student. This visit was mainly to review the 
scholarship requirements, discuss the student's schedule, and start cultivating the advisor- 
student relationship. 



22 This 77 1 to 1 ratio is for degree-seeking students. 
23 Habley (2004). 
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The Second Milestone: Midterm Evaluations 

For the second milestone, students were required to be enrolled in at least 12 credit 
hours (or 15 credit hours for the second through the fourth semesters), obtain midterm evalua- 
tions from each of their professors, be in good academic standing (defined as a GPA of 2.0 or 
higher), and meet with their VISTA advisor. To assist in getting the evaluations, a report was 
created that listed all the VISTA students and their instructors. Using this report, the advisors 
were able to send an electronic midterm evaluation to all the students’ instructors. If a professor 
did not complete the electronic midterm evaluation, the advisor used other options to obtain it 
— for example, providing a paper version of the midterm evaluation to their students to take to 
their instmctor(s) or sending an e-mail to the instructor(s) directly explaining why the evalua- 
tion was important and needed to be completed. 

Obtaining midterm evaluations proved to be a challenge for some students, in part be- 
cause the university has no official policy in place for professors to provide student evaluations 
at the midterm point. In addition, some courses did not have grades posted at the midterm 
evaluation period, meaning that the professor was not able to assess progress at that point. 
According to advisors, some students who did not obtain all their evaluations or knew that they 
were not meeting the requirements opted to skip the meeting with their advisors and just wait 
until the third milestone. 

Beyond the challenge of obtaining midterm reports from faculty, VISTA advisors faced 
other challenges in getting students to come in for the second milestone visit. Advisors e-mailed 
and called students to encourage them to come in and in some cases placed advising holds on 
the students’ academic records. This hold prevented the students from making changes to their 
schedules and barred them from obtaining their transcripts if they needed them. 

Advisors strongly encouraged students to attend the second milestone visit because this 
visit was seen as the most critical. If a student was performing poorly, this was the time that 
appropriate adjustments could be made to his or her schedule. For example, the advisor would 
discuss with the student the possibility of dropping a course and adding another course during 
the second 8-week session or taking a course during the winter break. In addition, this session 
would be a time to reinforce the need for the student to participate in tutoring services at the 
Center for Academic Program Support (CAPS) or with one of the Ethnic Centers (see Box 1). 

The Third Milestone: End of Semester 

The third milestone was the culmination of earning 12 credits (or 15 credits in the se- 
cond through fourth semesters) with at least a 2.0 GPA. Students were not required to meet with 
their advisors at this milestone. Advisors reviewed the student’s academic file to verify that the 
requirements had been meet, in which case the students would be notified that they could pick 
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Box 1 



Advising Services at the University of New Mexico 

The University of New Mexico system comprises four degree-granting colleges: Arts 
and Sciences, Engineering, Education, and Fine Arts and Architecture. However, most 
freshmen are initially placed in University College, a unit that serves as the academic 
home to students before they have declared a major. 

University College Advising. Students in University College receive advising from the 
University College Advisement Center, where they must meet with an advisor before 
they register for each semester. 

College Advising. Once students have declared a major (usually as sophomores), most 
receive advising from their degree-granting college. 

College Enric hm ent Program (CEP). This program targets first-generation, low- 
income, minority, and rural college students, as well as students with low ACT scores. 
CEP provides academic and financial aid advising, mentoring, and tutoring for students 
in remedial courses. CEP provides services to students throughout their college careers. 

Ethnic Centers. The American Indian Student Services, African American Student 
Services, and El Centro de la Raza provide advising, tutoring, and mentoring services to 
American Indian, African American, and Hispanic students, respectively, throughout 
their college careers. 



up their checks at the Cashier’ s Office or that funds were posted to their accounts. Despite the 
optional nature of this visit, many students still came in to meet with their advisors. During this 
visit, the advisor talked with the student about how the semester had gone, including what 
helped to make it a good or bad semester, and solidified the student’s schedule for the next 
semester. In addition, the advisor and student discussed whether the student needed to take an 
intersession course to recoup hours or GPA credits needed for the scholarship. 

In addition to the challenges that came with each milestone, challenges were faced in 
implementing VISTA overall . 24 For example, one challenge was the retention of VISTA 
advisors. High turnover, which is typical for the academic advising profession as a whole, 
meant that some students, instead of seeing one VISTA advisor for the duration of the scholar- 



24 A future report will discuss the program’s implementation in more detail. 
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ship period as intended, would have at least two different VISTA advisors. Another challenge 
was getting students to pick up their checks from the Cashier’ s Office. After each disbursement 
the site coordinator received a list of names of students who had not picked up their checks, and 
the advisors contacted them to provide reminders. 

Advising Visits 

Table 4 presents data on the number of visits that students in the VISTA group made to 
their advisors, using counseling records kept by the VISTA staff. UNM provides several 
advising programs to students, through various departments. For example, at any given time a 
student has the opportunity to receive advising from the University College Advisement Center, 
the College Enrichment Program, Ethnic Centers, and departmental advisors (see Box 1). Table 
4 presents data for advising through VISTA only. 

The table shows that 95 percent of the VISTA students saw an advisor at least once 
during the first semester. The majority of those visits were the required visits outlined by the 
VISTA Scholarship program. However, a substantial proportion of students saw their advisor 
more than three times — 39 percent saw their advisor four or more times — suggesting that 
these students developed a relationship with their VISTA advisor that went beyond the 
required visits. 

The bottom panel of Table 4 shows that students visited their advisors more frequently 
during the second semester. For example, 53.5 percent of students saw their advisor four or 
more times during the second semester, compared with 39 percent in the first semester. These 
data suggest that the VISTA advisors made connections with their students, or perhaps that the 
students were beginning to see the benefits of meeting with an advisor more often. 

As noted above, during the disbursement visits the advisor typically reviewed the stu- 
dent's progress and discussed the options of taking intersession courses and registering for 
additional support services, such as university-sponsored tutoring. During nondisbursement 
visits, advisors and students discussed a range of topics, including scheduling for the following 
semesters, requirements for the Lottery Scholarship, late-starting courses, additional financial 
aid, internships, and job opportunities on campus. 

Although a more formal analysis of student participation in advising sessions will be 
presented in the final report, the advisors noted some differences in the types of students who 
came in more versus less often. In particular, higher-achieving students tended to come in more 
often, while struggling students came in less often. Advisors reported that the lower-achieving 
students said they were embarrassed that they were not doing well and that they did not want to 
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The Performance-Based Scholarship Demonstration 
Table 4 



VISTA Advising Visits Among Program Group Members 
University of New Mexico 



Outcome 


Program 

Group 


First program semester 


Had one or more contacts with advisor (%) 


95.0 


Number of contacts (%) 


0 


5.0 


1 to 3 


56.0 


4 or more 


39.0 


Average number of contacts 


3.2 


Second pro cram semester 


Had one or more contacts with advisor (%) 


92.9 


Number of contacts (%) 


0 


7.1 


1 to 3 


39.4 


4 or more 


53.5 


Average number of contacts 


3.8 


Sample size 


536 



SOURCE: MDRC calculations from University of New Mexico VISTA advising data. 
NOTE: Rounding may cause slight discrepancies in sums and differences. 



let the advisor down. Other students reported that they did not come in for a visit because they 
knew that they could get the funds at the end of the semester during the third disbursement. 

In summary, the data on counseling visits suggest that students in the VISTA group did 
see their advisors, at a minimum to pick up the disbursement checks but often for more than 
that. The increase in visits during the second semester suggests that the students became more 
connected to or comfortable with their advisors and began to see the benefits of seeking guid- 
ance. Although the data suggest a fairly high level of advising for students in the VISTA group, 
it is not possible to examine whether students in the VISTA group received more counseling 
than students in the control group because control group students’ use of advising was not 
measured. A future report may use data on college counseling more generally to examine 
whether the VISTA program led to a counseling differential. 
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Scholarship Receipt and Financial Aid 

The VISTA scholarship enhances students' financial aid packages through at least two mecha- 
nisms. 25 First, the VISTA scholarship directly eases a student’s financial burden by increasing 
total financial resources available. During the first semester of the program, VISTA students 
received, on average, nearly $600 more in total financial aid than did students in the control 
group. During their second semester, VISTA students received more than $300 more in total 
financial aid. This effect is described below in “Effects on Financial Aid Package Composition.” 

In addition, some VISTA scholarship recipients replaced less attractive financial aid, 
particularly loans, with scholarship funds. Low-income students in particular may be averse to 
amassing large education debts. The reduced debt burden is described below under “Reduced 
Loan Burdens.” In both semesters, VISTA students’ financial aid packages included more 
scholarship and grant money and less borrowed money. The VISTA scholarship allowed 
students to reduce their debt burdens while earning richer total financial aid packages. 

The financial aid data reveal that second-semester academic milestones present a bigger 
hurdle than do those in the first semester. As a result, VISTA scholarship receipt is significantly 
less in the second semester than in the first. In addition, the VISTA scholarships lead to an 
increase in the total value of students' financial aid packages despite reduced borrowing by 
VISTA students. 

Receipt of VISTA Payments 

More than 90 percent of students who entered the VISTA program received at least one 
VISTA scholarship payment, and more than half — 54.9 percent — received the maximum 
scholarship amount of $2,000 during their first year in the program (Table 5). Whether or not 
payments are received is solely a matter of meeting the academic milestones and advising 
requirements. Table 5 allows an investigation of the role of academic milestones in VISTA 
scholarship receipt. VISTA students were more successful at earning milestone payments 
during the first semester than during the second semester, when more stringent credit require- 
ments were in place. 

Over the course of a semester, the milestones become more difficult to achieve, so the 
number of scholarship recipients declines. This decline is more pronounced in the second 
semester. During the first semester, at each successive milestone, the number of VISTA 
recipients declined by less than 7 percentage points. Just over 90 percent of VISTA students met 



25 If VISTA’s incentives or advising component improve academic performance, students may receive 
more non- VISTA merit aid — for example, the New Mexico State Lottery Scholarship. This indirect effect is 
not discussed in this section. 
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The Performance-Based Scholarship Demonstration 
Table 5 

VISTA Scholarship Receipt Among Program Group Members 
University of New Mexico 

Program 

Outcome Group 

First semester 

Received one or more scholarship payments (%) 90.9 

Received initial payment 90.9 

Received midterm payment 84. 1 

Received final payment (full $1,000) 77.6 

Number of scholarship payments received 2.5 

Average scholarship amount received ($) 826 

Average scholarship amount among recipients ($) 909 

Midterm payment recouped at end of semester (%) 23.5 

Secontl semester 

Received one or more scholarship payments (%) 81 .7 

Received initial payment 81.7 

Received midterm payment 69.4 

Received final payment (full $1,000) 58.4 

Number of scholarship payments received 2.1 

Average scholarship amount received ($) 671 

Average scholarship amount among recipients ($) 821 

Midterm payment recouped at end of semester (%) 18.1 

Cumulative ( over first two semesters) 

Received one or more scholarship payment in at least one semester (%) 92.9 

Received the full $ 1 ,000 in at least one semester 81.2 

Received the full $1,000 in each semester (total $2,000) 54.9 

Average scholarship amount received ($) 1,496 

Average scholarship amount among recipients ($) 1,610 

Midterm payment recouped at end of semester in at least one semester (%) 35.5 

Sample size 536 

SOURCE: MDRC calculations from University of New Mexico Performance Based Scholarships payment data. 

NOTES: Italics indicate statistics calculated from a subset of the full sample. 

During the first program semester, scholarship group members receive an initial payment of $250 after 
enrolling in 12 or more credits, a midterm payment of $250 if they are still enrolled in 12 or more credits and 
have a "C" or better average, and a final payment of $500 for completing 12 or more credits with a "C" or better 
average. During the second program semester, the scholarship benchmark goes up to 15 or more credits. 
Scholarship group members can make up credits during the intersession immediately following the semester in 
order to recoup missed payments. 
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the first milestone; 84.1 percent, or 6.8 percentage points fewer, received the midterm payment. 
More than three-fourths — 77.6 percent — received the final payment, including some who 
failed to satisfy the midterm requirement but met the final milestone. 

During the second semester, 81.7 percent of VISTA recipients met the first milestone, 
but only 69.4 percent — a 12.3 percentage point fall — met the requirements for the midterm 
payment. The decline from midterm to semester’s end was nearly as large: 11 percentage 
points. Only 58.4 percent of VISTA students received the final payment, including those who 
had not met the midterm milestone but met the final milestone. The share of students who 
received the full $1,000 in the second semester is almost 20 percentage points lower than the 
share who received the full $1,000 in the first semester. 

The sharp second semester decline in the number of students meeting each milestone 
and the decline in the proportion who were able to recover from failure to meet the midterm 
milestone combine to yield higher VISTA receipt during the first semester, and an average 
annual VISTA scholarship amount of just under $1,500. But, despite the challenges of the 
second semester, more than half of the VISTA students earned the full $2,000 during their first 
year of eligibility. 

Effects on Financial Aid Package Composition 

VISTA payments are made to students in addition to any other financial aid to which 
they are entitled. It is possible for a VISTA student to receive up to $2,000 more in total 
financial aid per year relative to a student in the control group. On average, VISTA students 
received approximately $1,500 in VISTA payments during their first year at UNM. They did 
not, though, receive total financial aid packages that were $1,500 more generous. 

Table 6 provides a detailed picture of the components of financial aid received by 
VISTA students and students in the control group. Eligibility for the Pell Grant is a condition 
for inclusion in the demonstration, so all students in both groups could have received a Pell 
Grant — and almost all did. In addition, as described above, entering students with a high 
school GPA of at least 2.5 qualify for the Bridge to Success Scholarship. This scholarship is 
awarded for the first semester of college, after which the New Mexico State Lottery Scholarship 
becomes available to continuing students with a UNM GPA of at least 2.5. Seventy-five percent 
of entering UNM freshmen qualify for the Bridge to Success Scholarship; more than 80 percent 
of VISTA and control group students received this funding. Combined, the Pell Grant and the 
Bridge to Success Scholarship provided, on average, almost $3,000 in financial assistance 
during the students’ first semester at UNM — more than half the students’ total aid packages. 

Patterns that were observed during the first semester remained true for the second se- 
mester (see second panel of Table 6): most students in both groups continued to receive a Pell 
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The Performance-Based Scholarship Demonstration 
Table 6 

Financial Assistance Among Sample Members, by Research Group 
University of New Mexico 



Outcome 


Program 

Group 


Control 

Group 


Difference 


Standard 

Error 


First program semester 


Received any financial assistance (%) 


98.5 


99.4 


-0.9 


0.6 


Pell Grant 


96.5 


98.3 


-1.9 * 


1.0 


UNM Bridge to Success Scholarship 


82.5 


84.2 


- 1.8 


2.3 


VISTA scholarship 


90.9 


0.0 


90.9 *** 


1.2 


Other grants ' 1 


71.1 


71.7 


-0.7 


2.8 


Subsidized loans 


28.5 


33.0 


-4.5 


2.8 


Unsubsidized loans 


14.7 


21.3 


-6.5 *** 


2.3 


Federal Work-Study * 1 


9.3 


10.1 


- 0.8 


1.8 


Total financial assistance received ($) 


5,847 


5,250 


597 *** 


131.1 


Pell Grant 


1,899 


1,965 


-66 


46.8 


UNM Bridge to Success Scholarship 


976 


999 


-23 


32.1 


VISTA scholarship 


826 


0 


826 *** 


14.5 


Other grants 3 


1,361 


1,319 


42 


105.3 


Subsidized loans 


451 


557 


-106 ** 


48.7 


Unsubsidized loans 


202 


256 


-54 


41.8 


Federal Work-Study * 1 


133 


154 


-21 


30.6 


Second program semester 


Received any financial assistance (%) 


92.4 


92.1 


0.2 


1.6 


Pell Grant 


91.0 


91.4 


-0.3 


1.7 


NM Lottery Scholarship 


71.6 


71.2 


0.4 


2.8 


VISTA scholarship 


80.2 


0.0 


80.2 *** 


1.7 


Other grants 3 


59.0 


62.2 


-3.2 


3.0 


Subsidized loans 


23.7 


30.1 


-6.4 ** 


2.7 


Unsubsidized loans 


13.1 


20.0 


_6 9 *** 


2.3 


Federal Work-Study * 1 


9.3 


11.7 


-2.4 


1.9 


Total financial assistance received ($) 


5,251 


4,917 


334 ** 


144.6 


Pell Grant 


1,788 


1,807 


-20 


53.8 


NM Lottery Scholarship 


1,417 


1,402 


15 


59.0 


VISTA scholarship 


615 


0 


615 *** 


17.6 


Other grants 3 


769 


839 


-71 


72.0 


Subsidized loans 


330 


474 


-144 *** 


43.3 


Unsubsidized loans 


196 


238 


-42 


42.6 


Federal Work-Study * 1 


137 


156 


-19 


29.7 


Sample size (total = 1,081) 


536 


545 







(continued) 
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Table 6 (continued) 



SOURCE: MDRC calculations from University of New Mexico financial aid data. 

NOTES: A two-tailed t-test was applied to differences between the research groups. Statistical significance levels 
are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

“Includes all grants and scholarships excluding the Pell Grant, NM Bridge to Success and Lottery grants, and 
VISTA scholarships. 

b Federal Work-Study amounts shown indicate the amount the student earned in the semester. 



Grant and to qualify for the Lottery Scholarship. During the second semester, just over 70 
percent of students in both the control and VISTA groups qualified for the Lottery Scholarship, 
fewer than had qualified for the Bridge to Success Scholarship. Those two grants — the Pell 
and the Lottery — continued to account for more than half the total value of students’ financial 
aid packages. VISTA students were slightly (1.9 percentage points) less likely to receive a Pell 
Grant in the first semester than were control group students. All other differences in Pell Grant 
and Bridge or Lottery Scholarship receipt were either small or statistically insignificant. 

Reduced Loan Burdens 

As shown in Table 6, during the first semester, VISTA and control group students were 
equally likely to have subsidized loans, but on average the control group students borrowed 
$106 more than did the VISTA students. VISTA students were less likely to have unsubsidized 
loans: more than 20 percent of control group students had unsubsidized loans but less than 15 
percent of VISTA students had such loans. 

While VISTA students received, on average, $826 in VISTA scholarship money during 
their first semester, the difference in total financial aid between the VISTA and control groups 
averaged $597, $229 less than would be the case if the VISTA funds were simply added to all 
other financial aid sources. Reduced borrowing accounted for more than 70 percent of this $229 
difference: VISTA students ended the semester with $160 less in loans relative to the control 
group students. 

Second semester results are even more striking. Borrowing is the only financial aid 
component, besides the VISTA scholarship, that differs significantly between VISTA students 
and those in the control group. VISTA students were less likely to have both unsubsidized loans 
(by 6.9 percentage points) and subsidized loans (by 6.4 percentage points). VISTA students’ 
average total financial assistance package during the second semester was $334 more than that 
for control group students; the average VISTA scholarship was $615. 26 Reductions in loan 



26 The average VISTA scholarship received in the second semester reported in Table 6 ($615) is somewhat 
less than that reported in Table 5 ($671) because the financial aid data used for Table 6 include records through 

(continued ) 
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burden among VISTA students account for more than 66 percent of the $280 difference 
between the VISTA scholarship and the overall difference in financial aid packages. On 
average, the VISTA students’ second semester debt burden was $186 less than that of the 
control group students. 

The reduction in loans may have been initiated by the students or the financial aid of- 
fice. In a few cases, for example, VISTA students’ other financial aid awards left less than 
$1,000 per semester remaining in unmet need. 27 In those cases, the students’ loans were reduced 
so that the student could receive the full VISTA scholarship. However, other analyses (not 
shown) suggest that the loan reduction was not all “automatic” repackaging by the financial aid 
office. Among students who entered the study with relatively high unmet need, who would still 
have $1,000 or more in unmet need even with the VISTA funds, VISTA also led to a reduction 
in loans. These students most likely made the decision to reduce their borrowing. 

In sum, more than 80 percent of VISTA students received the full $1,000 scholarship in 
at least one semester, and more than half received the full $2,000 during their first year. The 
average annual VISTA payment was just under $1,500, but the VISTA scholarship did not 
increase program students’ total financial aid by the full scholarship amount. Instead, the 
VISTA scholarship allowed students to reduce their debt burden. As a result, VISTA students’ 
financial aid packages were both more generous and included a higher proportion of grant and 
scholarship money than did the control group students’ packages. For a handful of students near 
the unmet need threshold, this result may have been forced by repackaged financial aid. How- 
ever, students with substantial unmet need also used the VISTA scholarship to reduce borrow- 
ing. VISTA students entered their second year at the university with lower accumulated debt. 



Early Look at Educational Impacts 

This section presents impacts on academic performance and enrollment status for the first two 
semesters of the program, the subsequent summer session, and initial enrollment in the third 
semester. 

Table 7, which presents academic outcomes in the first semester, shows only one statis- 
tically significant difference between program group and control group students. Students in the 



the spring of 2010, while the data used for Table 5 include records through the summer of 2010 and thus are 
more complete. 

27 UNM cannot offer financial aid in excess of a student's financial need, based on the difference between 
the estimated cost of attendance and the student's estimated family contribution as measured by the FAFSA. 
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The Performance-Based Scholarship Demonstration 
Table 7 



Academic Outcomes in the First Program Semester 
Among Sample Members, by Research Group 



University of New Mexico 



Outcome 


Program 

Group 


Control 

Group Difference 


Standard 

Error 


First nroeram semester 










Registered for any courses at 21 -day census (%) 


97.8 


99.0 


-1.2 


0.8 


Enrolled in any winter intersession courses 


12.0 


9.0 


3.0 


1.9 


Enrolled in any 8-week second-half courses 


13.3 


14.0 


-0.7 


2.1 


Average number of credits attempted 3 


14.8 


15.0 


-0.3 * 


0.2 


Regular credits 


13.3 


13.4 


-0.2 


0.2 


Developmental credits 


1.5 


1.6 


-0.1 


0.1 


Attempted 12 or more credits (%) 


96.9 


98.3 


-1.5 


0.9 


Attempted 15 or more credits (%) 


73.0 


75.8 


-2.8 


2.7 


Average number of credits earned 


12.8 


12.8 


0.0 


0.2 


Regular credits 


11.6 


11.5 


0.1 


0.2 


Developmental credits 


1.2 


1.3 


-0.1 


0.1 


Number of credits earned (%) 










1 8 or more 


4.2 


3.0 


1.3 


1.2 


15 to less than 18 


41.6 


42.8 


-1.2 


2.9 


12 to less than 15 


37.3 


36.0 


1.3 


2.9 


6 to less than 12 


8.8 


11.3 


-2.5 


1.8 


1 to less than 6 


2.2 


2.0 


0.2 


0.9 


No credits earned 15 


5.8 


4.9 


0.9 


1.4 


Passed all courses (%) 


64.1 


58.8 


- 


- 


Passed 80% or more of all courses attempted (%) 


85.3 


83.1 


- 


- 


Withdrew from any courses, if registered (%) 


20.7 


26.0 


- 


- 


Term GPA (%) 










3.0 to 4.33 


51.3 


52.1 


-0.7 


2.9 


2.5 to 2.9 


22.3 


19.9 


2.3 


2.5 


2.0 to 2.4 


9.4 


11.5 


-2.1 


1.9 


0.0 to 1.9 


13.3 


14.2 


-0.9 


2.1 


No GPA C 


3.7 


2.3 


1.4 


1.0 


Earned 15+ credits and earned a term GPA of 2.0 or greater (%) 


45.2 


45.1 


0.2 


2.9 


Earned 15+ credits and earned a term GPA of 2.5 or greater (%) 


43.1 


41.4 


1.7 


2.9 


Earned 12+ credits and earned a term GPA of 2.0 or greater (%) 


78.7 


77.6 


1.1 


2.5 


Earned 12+ credits and earned a term GPA of 2.5 or greater (%) 


71.0 


69.8 


1.3 


2.7 


Sample size (total = 1,081) 


536 


545 
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Table 7 (continued) 



SOURCE: MDRC calculations from University of New Mexico transcript data. 

NOTES: Rounding may cause slight discrepancies in sums and differences. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Estimates are adjusted by the following student characteristics: gender, race/ethnicity, mother and father's 
education levels, current employment, language spoken at home, high school GPA, and ACT composite score. 

Estimates for the first program semester include courses that were taken during winter intersession. 

Italics indicate statistics calculated only for students who registered for any classes. Because they are based on 
a subset of the full sample, impact estimates are not presented. 

“Credits attempted do not include those attempted at Central New Mexico Community College and 
recognized by UNM. These data were not available in time for this report, although they should only account for 
about 2 to 3 percent of total credits attempted. 

b The "No credits earned" category includes students who did not enroll and students who earned zero credits 
in their attempted courses. 

c The "No GPA" category includes students who did not enroll and students who took only developmental 
courses, which are not included in GPA calculations. 



program group registered for fewer credits (three-tenths of a credit less, on average), although 
the number of credits earned for the semester is the same for both groups. 

In terms of progress toward a degree, program and control group students completed 
the same number of credits in the first semester and there is no statistically significant difference 
between the groups in terms of the distribution of credits earned, GPA, and earned credit and 
GPA thresholds. 

Significant differences in academic progress, however, emerged in the second semester, 
as shown in Table 8. Program group students were less likely to be registered only part time (for 
fewer than 12 credits) and had registered for almost one credit more, on average. More striking- 
ly, 82.2 percent of program group students registered for 15 or more credits, compared with 
only 56.6 percent of control group students. This difference reflects the 15-credit hour program 
requirement that begins in the second semester. As mentioned earlier, the 15-credit hour 
requirement exceeds the 12 credits that are needed to be considered full time for financial aid 
reasons as well as for Lottery Scholarship eligibility. At the same time, a student who earns only 
12 credits a semester will take at least five and a half years to graduate. 

The total number of credits earned for the second semester is only slightly higher for 
program group students, who earned six-tenths of a credit more than control group students. 
This measure masks a much larger program effect in terms of the distribution of earned credits: 
program group students are much more likely — by 17.2 percentage points — to have earned 
15 to 17 credits, compared with students in the control group, who are significantly more likely 
to have earned 6 to 14 credits. Although program group students are much more likely than 
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The Performance-Based Scholarship Demonstration 
Table 8 

Academic Outcomes after the First Program Semester 
Among Sample Members, by Research Group 

University of New Mexico 

Program Control Standard 

Outcome Group Group Difference Error 



Second program semester 



Registered for any courses at 21 -day census (%) 


91.6 


91.2 


0.3 


1.7 


Enrolled part time (fewer than 12 credits) 


1.2 


2.8 


-1.6 * 


0.9 


Enrolled in any 8-week second-half courses 


23.5 


18.7 


4.8 * 


2.5 


Average number of credits attempted 11 


14.2 


13.4 


0 g *** 


0.3 


Regular credits 


13.9 


13.1 


q g *** 


0.3 


Developmental credits 


0.4 


0.3 


0.1 


0.1 


Attempted 12 or more credits (%) 


90.3 


88.2 


2.1 


1.9 


Attempted 15 or more credits (%) 


82.2 


56.6 


25.6 *** 


2.7 


Average number of credits earned 


11.7 


11.1 


0.6 * 


0.3 


Regular credits 


11.4 


10.9 


0.5 * 


0.3 


Developmental credits 


0.3 


0.2 


0.1 


0.0 


Number of credits earned (%) 










1 8 or more 


4.7 


4.9 


-0.2 


1.3 


15 to less than 18 


44.2 


27.0 


17 2 *** 


2.8 


12 to less than 15 


21.2 


32.2 


-10.9 *** 


2.7 


6 to less than 12 


14.1 


20.5 


_6 4 *** 


2.3 


1 to less than 6 


3.2 


2.7 


0.4 


1.0 


No credits eamed b 


12.6 


12.8 


-0.2 


2.0 


Passed all courses (%) 


S3. 8 


51.7 


- 


- 


Passed 80% or more of all courses attempted (%) 


75.4 


75.4 


- 


- 


Withdrew from any courses, if registered (%) 


30.6 


31.7 


- 


- 


Term GPA (%) 










3.0 to 4.33 


39.1 


42.3 


-3.2 


2.8 


2.5 to 2.9 


19.3 


19.6 


-0.3 


2.4 


2.0 to 2.4 


14.4 


11.4 


3.0 


2.0 


0.0 to 1.9 


18.1 


16.5 


1.6 


2.3 


No GPA C 


9.1 


10.3 


-1.2 


1.8 


Earned 15+ credits and earned a term GPA of 2.0 or greater (%) 


48.1 


31.7 


26 4 *** 


2.8 


Earned 15+ credits and earned a term GPA of 2.5 or greater (%) 


43.2 


30.0 


23 j *** 


2.7 


Earned 12+ credits and earned a term GPA of 2.0 or greater (%) 


66.1 


62.0 


4.0 


2.8 


Earned 12+ credits and earned a term GPA of 2.5 or greater (%) 


54.7 


55.3 


-0.6 


2.9 
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Table 8 (continued) 





Program 


Control 




Standard 


Outcome 


Group 


Group Difference 


Error 


Summer term 










Registered for any courses at 21 -day census (%) 


27.4 


22.2 


5.2 


** 2.6 


Cumulative (over first two urogram semesters and summer term) 










Average number of credits earned 


25.6 


24.8 


0.8 


0.5 


Earned 27+ credits and earned a cumulative GPA of 2.0 or greater (%) 


63.2 


56.2 


7.1 


** 2 9 


Earned 30+ credits and earned a cumulative GPA of 2.0 or greater (%) 


42.8 


34.0 


8.8 


*** 9 g 


Third urogram semester 










Registered for any courses at 21 -day census (%) 


77.3 


78.2 


-0.9 


2.5 


Average number of credits attempted 


12.1 


11.5 


0.6 


0.4 


Regular credits 


12.0 


11.4 


0.6 


0.4 


Developmental credits 


0.1 


0.1 


-0.1 


0.0 


Attempted 12 or more credits (%) 


75.6 


73.5 


2.1 


2.6 


Attempted 15 or more credits (%) 


66.8 


51.0 


15.7 


*** 2 9 


Sample size (total = 1,081) 


536 


545 







SOURCE: MDRC calculations from University of New Mexico transcript data. 



NOTES: Rounding may cause slight discrepancies in sums and differences. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Estimates are adjusted by the following student characteristics: gender, race/ethnicity, mother and father's 
education levels, current employment, language spoken at home, high school GPA, and ACT composite score. 

Italics indicate statistics calculated only for students who registered for any classes. Because they are based on 
a subset of the full sample, impact estimates are not presented. 

a Credits attempted do not include those attempted at Central New Mexico Community College and recognized 
by UNM. These data were not available in time for this report, although they should only account for about 2 to 3 
percent of total credits attempted. 

b The "No credits earned" category includes students who did not enroll and students who earned zero credits 
in their attempted courses. 

c The "No GPA" category includes students who did not enroll and students who took only developmental 
courses, which are not included in GPA calculations. 



control group students to attempt 15 or more credits, students in both groups earned comparable 
grades. At the same time, program group students were 16.4 percentage points (or 52 percent) 
more likely than control group students to earn 15 or more credits with a GPA of 2.0 or greater 
(the program requirement) and 13.1 percentage points (or 44 percent) more likely to earn 15 or 
more credits with a GPA of 2.5 or greater. Program and control group students were equally 
likely to earn 12 or more credits with a GPA of 2.5 or greater, which is the requirement for the 
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Lottery Scholarship. This suggests that VISTA did not have a negative effect on receipt of the 
Lottery Scholarship. 

Given the higher credit requirements for program group students to retain their VISTA 
scholarship, the program might lead to greater use of courses offered in the second half of the 
semester and in the summer. Recall that VISTA students qualify for the final semester payment 
if they recover credits in the winter intersession or summer session. In fact, program group 
students were about 5 percentage points more likely to take those courses than were control 
group students. 

By the end of the summer semester, students in the program group were significantly 
more likely — by 7.1 percentage points — to have completed at least 27 credits (the VISTA 
requirements) in the first and second semesters with a cumulative GPA of 2.0 or higher, and 
they were 8.8 percentage points more likely to have completed a total of 30 credits with a GPA 
of 2.0 or higher. Those differences are markedly smaller than the 16.4 percentage point gap 
between program and control group students who completed 15 or more credits with a 2.0 or 
higher GPA in the second semester, which suggests that not all students who met VISTA 
criteria in the second semester also met them in the first semester. Nevertheless, the size of the 
difference is large relative to the 9.8 percentage point difference in second semester progress to 
degree between Pell-eligible and all UNM students shown earlier in Table 1. At least initially, 
VISTA appears to have narrowed the gap in progress between economically disadvantaged 
students and their more affluent peers. 

It appears that VISTA did not affect enrollment of students into the third semester. The 
last panel of Table 8 shows no significant differences in third semester enrollment for VISTA 
and control group students. There is also no significant difference in the average number of 
credits attempted between the program and control group students. Nevertheless, program 
group students were more likely — by 15.7 percentage points (or 31 percent) — to have 
registered for 15 or more credits. Thus, VISTA students continued to be more likely than 
control group students to stay on track for graduation. 

Although the 15-credit requirement appears to be driving these outcomes, there may be 
more to the story. The program may also have affected performance by reducing students’ 
reliance on loans. Alternatively, the enhanced counseling may have helped students focus on 
earning enough credits to make timely progress toward a degree. 



Conclusions 

The early findings on the VISTA program are modest but encouraging. Students who were 
eligible for the VISTA scholarship succeeded in earning VISTA payments, visited their VISTA 
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advisor fairly often during the year, and used some of their VISTA payments to reduce their 
borrowing. By the end of their first year in college, VISTA students were more likely than 
control group students to have earned 30 credits, increasing the likelihood that they would be on 
track for an on-time graduation. However, the overall effect on credits earned was fairly small. 

VISTA students are eligible for the program for two more semesters. It will be im- 
portant to examine whether the program' s effects in the third and fourth semesters build on the 
effects that have already been observed. However, there might be reason to expect additional 
effects even if the scholarship were not offered beyond Year 1 . First, as observed above, VISTA 
students seem to have visited their advisors fairly regularly. Although the data for this report do 
not indicate whether they visited academic advisors more than their control group counterparts 
did, it is possible that the quantity and quality of VISTA advising will lead to lasting positive 
effects on student performance. Second, some students used the VISTA payments to reduce 
their reliance on loans. Studies suggest that the reluctance to accumulate debt can be an im- 
portant deterrent to enrolling in and completing college. 28 Finally, although the overall effect on 
credits earned is small, it is possible that getting and staying on track early can have lasting 
positive effects. 

A report on the longer-term findings from the PBS Demonstration at UNM will be re- 
leased in 2014. In addition to including effects on progress and performance through eight 
semesters, the report will include findings from an in-depth implementation study, focus groups 
with students, and a survey of students on their use of time during the semester. 



28 Burdman (2005). 
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Selected Characteristics of Sample Members at Baseline, 

by Research Group 




The Performance-Based Scholarship Demonstration 
Appendix Table A.l 

Selected Characteristics of Sample Members at Baseline, by Research Group 



University of New Mexico 



Characteristic 


Full 

Sample 


Program 

Group 


Control 

Group 


Female (%) 


60.8 


61.4 


60.2 


Age (%) 


17-18 years old 


93.7 


94.4 


93.0 


19-20 years old 


6.3 


5.6 


7.0 


Marital status (%) 


Married 


0.6 


0.6 


0.7 


Unmarried 


90.5 


91.0 


89.9 


Missing 


8.9 


8.4 


9.4 


Has one or more children (%) 


1.8 


1.7 


1.8 


Race/ ethnicity 11 (%) 


Hispanic 


60.6 


60.2 


61.0 


White 


21.8 


21.5 


22.2 


Black 


2.7 


3.2 


2.2 


Asian or Pacific Islander 


3.5 


3.2 


3.9 


Native American 


6.9 


6.9 


6.8 


Other* 5 


4.5 


5.0 


3.9 


Language other than English spoken regularly in home (%) 


22.0 


20.8 


23.2 


Diplomas/degrees earned 1 (%) 


High school diploma 


97.8 


97.2 


98.3 


GED certificate 


1.3 


1.9 


0.7 


Other 


1.2 


1.3 


1.1 


First person in family to attend college (%) 


32.8 


32.1 


33.5 


ACT English'* 


25th percentile score 


17 


16 


17 


75th percentile score 


23 


24 


23 


ACT Math” 


25th percentile score 


16 


16 


17 


75th percentile score 


23 


23 


23 


High school cumulative GPA 


3.3 


3.3 


3.3 


High school cumulative GPA (%) 


3.5 to 4.4 


38.2 


39.7 


36.7 


3 to less than 3.5 


33.9 


32.6 


35.0 


2 to less than 3 


24.6 


24.4 


24.8 


0 to less than 2 


0.0 


0.0 


0.0 


No GPA available 


3.3 


3.2 


3.5 
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Appendix Table A.l (continued) 



Characteristic 



Full Program Control 

Sample Group Group 



Highest degree/diploma earned by father (%) 



Not a high school graduate 


15.6 


15.1 


16.1 


High school diploma or GED certificate 


37.3 


40.1 


34.5 


Some college or associate's degree 


17.9 


13.6 


22.0 


Bachelor's degree or higher 


14.4 


16.0 


12.8 


Missing 


14.8 


15.1 


14.5 


Highest degree/diploma earned by mother (%) 


Not a high school graduate 


11.9 


11.6 


12.3 


High school diploma or GED certificate 


36.8 


37.7 


36.0 


Some college or associate's degree 


24.1 


22.8 


25.3 


Bachelor's degree or higher 


19.7 


20.1 


19.3 


Missing 


7.5 


7.8 


7.2 


Currently working (%) 


49.0 


49.4 


48.5 


Among those currently working, hours worked per week (%) 


1-10 hours 


5.9 


3.0 


8.8 


11-20 hours 


33.3 


36.5 


30.2 


21-30 hours 


29.9 


32.7 


27.1 


31-40 hours 


26.3 


23.6 


29.0 


More than 40 hours 


4.6 


4.2 


5.0 


Average hourly wage ($) 


8.2 


8.2 


8.3 


Plans to live on campus (%) 


42.9 


41.8 


44.0 


Sample Size 


1,081 


536 


545 



SOURCE: MDRC calculations using Baseline Information Form (BIF) data and University of New Mexico 
placement test and high school data. 

NOTES: To analyze whether baseline characteristics jointly predicted research group status, a likelihood ratio test 
was performed. This yielded a p-value of 0.411. Convention suggests that this probability of differences occuring 
by chance is large enough that these differences can be ignored in the analyses. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Italics indicate statistics calculated from a subset of the full sample. 

Missing values are not included in individual variable distributions. 

Distributions may not add to 100 percent because of rounding. 

“Respondents who said they are Hispanic and chose a race are included only in the Hispanic category. 
Respondents who said they are not Hispanic and chose more than one race are included in the Other category. 
b Other includes multiracial and other races/ethnicities. 

“Distributions may not add to 100 percent because categories are not mutually exclusive. 
d ACT scores range from 1 to 36. The median test taker who graduated from high school between 2008 and 
2010 earned a 20 in both the English and Math sections. The 25th percentile score was 15 for English and 16 for 
Math, and the 75th percentile score was 24 for both subjects. See http://www.actstudent.org/scores/normsl.html. 



34 








References 



Baum, Sandy, and Jennifer Ma. 2010. Trends in College Pricing 2010. New York: The College 
Board. 

Baum, Sandy, and Marie O’Malley. 2003. “College on Credit: How Borrowers Perceive Their 
Education Debt.” NASFAA Journal of Student Financial Aid 33, no. 3: 7-19. 

Bettinger, Eric. 2004. “How Financial Aid Affects Persistence in College.” Pages 207-233 in C. M. 
Hoxby (ed.), College Choices: The Economics of Where to Go, When to Go, and How to Pay 
for It. Chicago: University of Chicago Press. 

Burdman, Pamela. 2005. The Student Debt Dilemma: Debt Aversion as a Barrier to College Access. 
Berkeley: Center for Students in Higher Education, University of California. 

Dynarski, Susan. 2008. “Building the Stock of College-Educated Labor.” Journal of Human 
Resources 43, no. 3: 576-610. 

Dynarski. Susan, and Judith E. Scott-Clayton. 2008. “Complexity and Targeting in Federal Student 
Aid: A Quantitative Analysis.” Tax Policy and the Economy 22 (July): 109-150. 

Habley, Wes. 2004. “Advisor Load.” NACADA Clearinghouse of Academic Advising Resources. 
Web site: www.nacada.ksu.edu/Clearinghouse/AdvisingIssues/advisorload.htm. 

Long, Bridget Terry. 2008. What Is Known About the Impact of Financial Aid? Implications for 
Policy. New York: National Center for Postsecondary Research. 

Richburg-Hayes, Lashawn, Thomas Brock. Allen LeBlanc, Christina Paxson, Cecilia Elena Rouse, 
and Lisa Barrow. 2009a. Rewarding Persistence: Effects of a Performance-Based Scholarship 
Program for Low-Income Parents. New York: MDRC. 

Richburg-Hayes, Lashawn, Paulette Cha, Monica Cuevas, Amanda Grossman, Reshrna Patel, and 
Colleen Sommo. 2009b. Paying for College Success: An Introduction to the Performance- 
Based Scholarship Demonstration. New York: MDRC. 

Scott-Clayton, Judith. 2009. On Money and Motivation: A Quasi-Experimental Analysis of 
Financial Incentives for College Achievement. New York: Columbia University, Teachers 
College. 

Snyder, Thomas D., and Sally A. Dillow. 2010. Digest of Education Statistics, 2010. NCES 
01 1015. Washington, DC: National Center for Education Statistics. 

Thaler, Richard. 1999. “Mental Accounting Matters.” Journal of Behavioral Decision Making 12 
(December): 183-206. 

Turner, Sarah. 2004. “Going to College and Finishing College: Explaining Different Educational 
Outcomes.” In C. Hoxby (ed.), College Choices: The Economics of Where to Go, When to Go, 
and How to Pay for It. Chicago: University of Chicago Press. 

Zernike, Kate. 2010. “Annual Poll of Freshmen Shows Effect of Recession.” New York Times, 
January 21, p. A25. 



35 




Earlier MDRC Publications on the Performance-Based 
Scholarship Demonstration 



Promoting Full-Time Attendance Among Adults in Community College 

Early Impacts from the Performance-Based Scholarship Demonstration in New York 

2011. Lashawn Richburg-Hayes, Colleen Sommo, Rashida Welbeck. 

Rewarding Progress , Reducing Debt 

Early Results from Ohio ’s Performance-Based Scholarship Demonstration for Low-Income 
Parents 

2010. Paulette Cha and Reshma Patel. 



NOTE: All MDRC publications are available for free download at www.mdrc.org. 
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About MDRC 



MDRC is a nonprofit, nonpartisan social policy research organization dedicated to learning 
what works to improve the well-being of low-income people. Through its research and the 
active communication of its findings, MDRC seeks to enhance the effectiveness of social and 
education policies and programs. 

Founded in 1974 and located in New York City and Oakland, California, MDRC is best known 
for mounting rigorous, large-scale, real-world tests of new and existing policies and programs. 
Its projects are a mix of demonstrations (field tests of promising new program approaches) and 
evaluations of ongoing government and community initiatives. MDRC’s staff bring an unusual 
combination of research and organizational experience to their work, providing expertise on the 
latest in qualitative and quantitative methods and on program design, development, implementa- 
tion, and management. MDRC seeks to learn not just whether a program is effective but also 
how and why the program’s effects occur. In addition, it tries to place each project’s findings in 
the broader context of related research — in order to build knowledge about what works across 
the social and education policy fields. MDRC’s findings, lessons, and best practices are proac- 
tively shared with a broad audience in the policy and practitioner community as well as with the 
general public and the media. 

Over the years, MDRC has brought its unique approach to an ever-growing range of policy 
areas and target populations. Once known primarily for evaluations of state welfare-to-work 
programs, today MDRC is also studying public school reforms, employment programs for ex- 
offenders and people with disabilities, and programs to help low-income students succeed in 
college. MDRC’s projects are organized into five areas: 

• Promoting Family Well-Being and Child Development 

• Improving Public Education 

• Promoting Successful Transitions to Adulthood 

• Supporting Low-Wage Workers and Communities 

• Overcoming Barriers to Employment 

Working in almost every state, all of the nation’s largest cities, and Canada and the United 
Kingdom, MDRC conducts its projects in partnership with national, state, and local govern- 
ments, public school systems, community organizations, and numerous private philanthropies. 




